Method of controlling image acquisition and other related tools

ABSTRACT

The present disclosure provides a method for image capturing, a method for image capturing control and associated apparatuses. The method for image capturing includes: obtaining Region Of Interest (ROI) information of a target ROI corresponding to a current frame of image; buffering captured pixel data of the current frame of image; obtaining, when determining that the pixel data of the current frame of image has been buffered completely, pixel data associated with the target ROI from the buffered pixel data of the current frame of image based on the ROI information if the ROI information of the target ROI corresponding to the current frame of image can be obtained; and outputting the pixel data associated with the target ROI.

PRIORITY CLAIM AND RELATED APPLICATION

This U.S. patent document claims the priority of and the benefits ofChinese Patent Application No. 201810076488.4 of the same title andcontent that was filed by Applicant Beijing Tusen Weilai Technology Co.,Ltd. at the State Intellectual Property Office of China (SIPO) on Jan.26, 2018, which was published under the Publication No. 108307113 onJul. 20, 2018.

TECHNICAL FIELD

The present disclosure relates to image capturing technology, and moreparticularly, to a method and an apparatus for image capturing, a methodand an apparatus for image capturing control, a system for imagecapturing and a camera.

BACKGROUND

With the development of technical fields such as unmanned aerialvehicles, unmanned ships, self-driving, Virtual Reality (VR),three-dimensional (3D) scanning, robotic obstacle avoidance andSimultaneous Localization And Mapping (SLAM), cameras are widely used inthese technical fields. Since these fields have high requirements on acamera' resolution, in the related art, a camera having high resolution(for example, 4096*4096) is typically used for capturing images.However, due to such high resolution, the amount of data to betransmitted by the camera is so large that the transmission speed of theimage data is relatively slow. Accordingly, how to capture image datawith high resolution while improving its transmission speed has become atechnical problem to be solved by those skilled in the art.

In addition, in order to adapt image data outputted by a camera to humaneyes, Image Signal Processing (ISP) is typically integrated in thecamera, as shown in FIG. 1A, or provided separately from the camera, asshown in FIG. 1B. An image sensor of the camera transmits captured pixeldata to the ISP for processing, and the ISP processes the pixel databefore outputting it. However, in fact the ISP may lose some detailinformation in the pixel data when processing the pixel data. Hence, theimage data processed by the ISP may still have a problem of informationloss to some extent.

SUMMARY

According to embodiments of the present disclosure, a method for imagecapturing is provided. The method includes obtaining Region Of Interest(ROI) information of a target ROI corresponding to a current frame ofimage. The method also includes buffering captured pixel data of thecurrent frame of image. The method also includes obtaining, whendetermining that the pixel data of the current frame of image has beenbuffered completely, pixel data associated with the target ROI from thebuffered pixel data of the current frame of image based on the ROIinformation if the ROI information of the target ROI corresponding tothe current frame of image can be obtained. The method also includesoutputting the pixel data associated with the target ROI.

According to some embodiments of the present disclosure, a method forimage capturing control is provided. The method includes predicting aposition region in which a target object is to be located in a nextframe of image based on an object tracking result associated with aprevious frame of image, and determining the position region as a targetROI corresponding to the next frame of image. The method also includestransmitting ROI information of the target ROI corresponding to the nextframe of image to a camera.

According to some embodiments of the present disclosure, a camera isprovided. The camera includes an image sensor configured to capturepixel data of a current frame of image in accordance with apredetermined camera parameter and transmit the captured pixel data toan apparatus for image capturing. The camera also includes an apparatusfor image capturing configured to obtain ROI information of a target ROIcorresponding to the current frame of image; receive the pixel data ofthe current frame of image from the image sensor and buffer the pixeldata; obtain, when determining that the pixel data of the current frameof image has been buffered completely, pixel data associated with thetarget ROI from the buffered pixel data of the current frame of imagebased on the ROI information if the ROI information of the target ROIcorresponding to the current frame of image can be obtained; and outputthe pixel data associated with the target ROI.

According to some embodiments of the present disclosure, a system forimage capturing is provided. The system includes an apparatus for imagecapturing control configured to predict a position region in which atarget object is to be located in a next frame of image based on anobject tracking result associated with a previous frame of image, anddetermine the position region as a target ROI corresponding to the nextframe of image; and transmit ROI information of the target ROIcorresponding to the next frame of image to a camera. The system alsoincludes a camera configured to obtain ROI information of a target ROIcorresponding to a current frame of image from the apparatus for imagecapturing control; capture pixel data of the current frame of image inaccordance with a predetermined camera parameter and buffer the pixeldata of the current frame of image; obtain, when determining that thepixel data of the current frame of image has been buffered completely,pixel data associated with the target ROI from the buffered pixel dataof the current frame of image based on the ROI information if the ROIinformation of the target ROI corresponding to the current frame ofimage can be obtained; and output the pixel data associated with thetarget ROI.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are schematic diagrams each showing a structure of acamera and ISP in the related art;

FIG. 2 is a flowchart illustrating a method for image capturingaccording to some embodiments of the present disclosure;

FIGS. 3A and 3B are schematic diagrams each showing a target ROIcorresponding to a frame of image according to embodiments of thepresent disclosure;

FIG. 4 is a flowchart showing a specific process for Step 103 accordingto some embodiments of the present disclosure;

FIGS. 5A, 5B and 5C are schematic diagrams each showing a positionregion and ROI information corresponding to a target ROI;

FIG. 6 is another flowchart illustrating a method for image capturingaccording to some embodiments of the present disclosure;

FIGS. 7A, 7B and 7C are schematic diagrams each showing n pieces ofadjacent pixel data according to some embodiments of the presentdisclosure;

FIG. 7D is a schematic diagram showing a frame of image according tosome embodiments of the present disclosure;

FIGS. 7E and 7F are schematic diagrams each showing combination of pixeldata in the frame of image shown in FIG. 7D according to someembodiments of the present disclosure;

FIG. 7G is a schematic diagram showing combination of a plurality ofpieces of pixel data into one according to some embodiments of thepresent disclosure;

FIG. 8 is a first flowchart illustrating a method for image capturingcontrol according to some embodiments of the present disclosure;

FIG. 9 is another flowchart illustrating a method for image capturingcontrol according to some embodiments of the present disclosure;

FIG. 10A is a schematic diagram showing an object tracking resultassociated with a previous frame of image according to some embodimentsof the present disclosure;

FIG. 10B is a schematic diagram showing a target ROI corresponding to anext frame of image according to some embodiments of the presentdisclosure;

FIG. 11A is a schematic diagram showing an object detection resultassociated with a current frame of image according to some embodimentsof the present disclosure;

FIG. 11B is a schematic diagram showing an object tracking resultassociated with a previous frame of image according to some embodimentsof the present disclosure;

FIG. 11C is a schematic diagram showing an object tracking resultassociated with a current frame of image according to some embodimentsof the present disclosure;

FIG. 12 is a schematic diagram showing a structure of an apparatus forimage capturing according to some embodiments of the present disclosure;

FIG. 13 is a schematic diagram showing a structure of a camera accordingto some embodiments of the present disclosure;

FIG. 14 is a schematic diagram showing a structure of an apparatus forimage capturing control according to some embodiments of the presentdisclosure;

FIG. 15 is another schematic diagram showing a structure of an apparatusfor image capturing control according to some embodiments of the presentdisclosure; and

FIG. 16 is a schematic diagram showing a structure of a system for imagecapturing according to some embodiments of the present disclosure.

DETAILED DESCRIPTION OF THE EMBODIMENTS

In the following, the solutions according to the embodiments of thepresent disclosure will be described clearly and completely withreference to the figures. Obviously, the embodiments described below areonly some, rather than all, of the embodiments of the presentdisclosure. All other embodiments that can be obtained by those skilledin the art based on the embodiments described in the present disclosurewithout any inventive efforts are to be encompassed by the scope of thepresent disclosure.

The present disclosure provides a method and an apparatus for imagecapturing, a method and an apparatus for image capturing control, asystem for image capturing and a camera, capable of solving the problemthat the existing image capturing technique cannot achieve both highresolution and high transmission speed.

With the solutions according to the embodiments of the presentdisclosure, when a frame of image is captured, pixel data associatedwith the frame of image is buffered first. Then, pixel data associatedwith a target ROI corresponding to the frame of image may be obtainedfrom the buffered pixel data of the frame of image based on ROIinformation of the target ROI and may be transmitted. That is, with thesolutions according to the present disclosure, on one hand, pixel datamay be captured with a high-resolution camera to obtain detailinformation of each piece of pixel data. On the other hand, when theimage data is transmitted, instead of transmitting the pixel data of theentire frame of image, only pixel data associated with a target ROI,which is typically useful to a party requiring the image data, istransmitted. In this way, the amount of data to be transmitted may bereduced and thus the transmission speed of the image data may beincreased. Furthermore, in the solutions according to the presentdisclosure, the pixel data captured by the camera does not need to besubjected to ISP and thus no detail information of the pixel data willbe lost. As all the detail information of the pixel data may bereserved, the accuracy and integrity of the pixel data may be improved.As no ISP is required, the time required for ISP may be saved, whichfurther increases the transmission speed of the image data. Therefore,with the solutions according to the present disclosure, thehigh-resolution image data may be provided to the party requiring theimage data, while increasing the transmission speed of the image data,reserving all the detail information of the image data and improving theaccuracy of the image data.

The solutions according to the embodiments of the present disclosure maybe applied to an unmanned vehicle (including a truck, a publictransportation bus, a bus, a passenger car, a tractor, a sprinkler, agarbage truck, etc.), an unmanned aerial vehicle, an unmanned ship, arobot or the like. The present disclosure is not limited to any specificapplication scenario of the technical solutions.

Embodiment 1

According to Embodiment 1 of the present disclosure, a method for imagecapturing is provided. The method is applicable to a camera. As shown inthe flowchart of FIG. 2, the method includes the following steps.

At step 101, Region Of Interest (ROI) information of a target ROIcorresponding to a current frame of image is obtained.

At step 102, captured pixel data of the current frame of image isbuffered.

At step 103, when determining that the pixel data of the current frameof image has been buffered completely, it is determined whether the ROIinformation of the target ROI corresponding to the current frame ofimage may be obtained. If so, the method proceeds with step 104.

At step 104, pixel data associated with the target ROI is obtained fromthe buffered pixel data of the current frame of image based on the ROIinformation.

At step 105, the pixel data associated with the target ROI is outputted.

In Embodiment 1 of the present disclosure, the steps 101 and 102 are notnecessarily performed in a particular order. For example, the step 101may be performed before, after, or simultaneously with the step 102.

Assuming that a frame of image has a resolution of 4096*4096, the frameof image contains 4096 rows by 4096 columns of pixel data. In the step103, as a non-limiting example, it may be determined whether the pixeldata of the current frame of image has been buffered completely by 1)determining that the pixel data of the current frame of image has beenbuffered completely when a frame validity signal for the current frameof image ends; or 2) determining that the pixel data of the currentframe of image has been buffered completely when 4096*4096 pieces ofpixel data have been received in a valid period of the frame validitysignal for the current frame of image.

In some embodiments of the present disclosure, there may be one or moretarget ROIs corresponding to one frame of image. The number of targetROIs contained in each frame of image may or may not be the same. Thereis one target ROI corresponding to one frame of image, as shown in FIG.3A, or three ROIs corresponding to one frame of image, as shown in FIG.3B.

In some embodiments, the step 104 of obtaining the pixel data associatedwith the target ROI from the buffered pixel data of the current frame ofimage based on the ROI information may be implemented according to theprocess shown in FIG. 4, which includes the following steps.

At step a, the pixel data associated with the target ROI in the pixeldata of the current frame of image is determined based on the ROIinformation.

At step b, a storage address of the pixel data associated with thetarget ROI in a buffer is determined.

At step c, the pixel data is read from the storage address to obtain thepixel data associated with the target ROI.

In some embodiments of the present disclosure, each target ROI for oneframe of image may correspond to one piece of ROI information, or aplurality of target ROIs for one frame of image may correspond to onepiece of ROI information. The ROI information contains informationdescribing a position region of the target ROI in the frame of image.For example, assuming that the target ROI is a rectangle, the ROIinformation corresponding to the ROI may contain a position of astarting vertex (denoted as (x0, y0)), a width (denoted as W) and aheight (denoted as H) of the rectangle, and the position region of thetarget ROI is shown in FIG. 5A. Alternatively, the ROI informationcorresponding to the ROI may contain a position of a starting vertex anda position of an end point (denoted as (x, y)) of the rectangle, and theposition region of the target ROI is shown in FIG. 5B. The pixel datawithin the regions shown in FIGS. 5A and 5B, including the pixel data atthe border of the regions, is the pixel data associated with the targetROI. As another example, assuming the target ROI is a circle, the ROIinformation corresponding to the ROI may contain a position of an origin(denoted as (x0, y0)) and a radius (denoted as r) of the circle, and theposition region of the target ROI is shown in FIG. 5C. The pixel datawithin the region shown in FIG. 5C, including the pixel data at theborder of the region, is determined as the pixel data associated withthe target ROI. Of course, the shape of the position region of thetarget ROI may be other geometric shapes, e.g., a triangle, a square oran ellipse, to name a few.

In embodiments of the present disclosure, a fixed storage space may beallocated in advance for buffering the pixel data of one frame of image.The pixel of data of the current frame of image may be storedsequentially in the storage space in an outputting order. When the pixeldata associated with the target ROI corresponding to the current frameof image has been outputted, the buffered pixel data of the currentframe of image may be removed from the storage space, such that pixeldata of the next frame of image may be buffered in the storage space.

In some embodiments of the present disclosure, pixel points in one frameof image may be indexed in advance and a correspondence between theindices and storage addresses at which pixel data corresponding to therespective indices is stored in the storage space may be established. Inthe step b, the storage addresses of the respective pieces of pixel datamay be determined from the correspondence based on the indices of therespective pieces of pixel data associated with the target ROI.

Of course, it may be appreciated by those skilled in the art that thepixel data associated with the target ROI may be obtained from thestorage space in alternative ways. For example, a storage address ofstarting pixel data of one frame of image and address offsets of otherrespective pixel points with respect to the starting pixel point may bestored in advance. When the pixel data associated with the target ROI isdetermined, the following steps may be performed for each piece of pixeldata: moving a pointer from the storage address of the starting pixeldata, as a starting address, to the storage address of that piece ofpixel data by the address offset of that piece of pixel data withrespect to the starting pixel data; and reading the pixel data at thestorage address.

In practice, there may be a case where a certain frame of image does nothave any ROI information of its corresponding target ROI. In order totransmit the image data of the frame of image successfully and increasethe transmission speed of the image data of the frame of image, in anembodiment of the present disclosure, the above process shown in FIG. 2may further include steps 106 and 107, as shown in FIG. 6.

The step 103 may further include: proceeding with step 106 otherwise.

At step 106, n*m pieces of adjacent pixel data in the pixel data of thecurrent frame of image are combined into one piece of pixel datasequentially, to obtain compressed pixel data associated with thecurrent frame of image, where n is a natural number divisible by a totalnumber of rows in the current frame of image, and m is a natural numberdivisible by a total number of columns in the current frame of image.

At step 107, the compressed pixel data is outputted.

In some embodiments of the present disclosure, n may have a value of 1and m may be a natural number having a value larger than 1. In thiscase, the n*m pieces of adjacent pixel data in the above step 106 may bem pieces of horizontally consecutive pixel data, as shown in FIG. 7A(where each square represents one piece of pixel data). In an embodimentof the present disclosure, m may have a value of 1 and n may be anatural number having a value larger than 1. In this case, the n*mpieces of adjacent pixel data in the above step 106 may be n pieces ofvertically consecutive pixel data, as shown in FIG. 7B. In an embodimentof the present disclosure, each of n and m may be a natural numberlarger than 1 and the n*m pieces of adjacent pixel data in the abovestep 106 may constitute a rectangle or square, as shown in FIG. 7C. Thevalues of n and m may be set by those skilled in the art as appropriatedepending on actual requirements. The present disclosure is not limitedto any specific values of n and m.

It is assumed that one frame of image contains 4096*4096 pieces of pixeldata, as shown in FIG. 7D (where each square represents one piece ofpixel data). When each of m and n has a value of 2, four pieces ofadjacent pixel data in the pixels of the one frame of image may becombined into one piece of pixel data, resulting in 2048*2048 pieces ofcompressed pixel data, as shown in FIG. 7E (where the four hatchedsquares on the left represent four pieces of adjacent pixel data, andthe one hatched square on the right represents one piece of pixel dataobtained by combining the four pieces of adjacent pixel data). When nhas a value of 2 and m has a value of 4, eight pieces of adjacent pixeldata in the pixels of the one frame of image may be combined into onepiece of pixel data, resulting in 2048*1024 pieces of compressed pixeldata, as shown in FIG. 7F (where the eight hatched squares on the leftrepresent eight pieces of adjacent pixel data, and the one hatchedsquare on the right represents one piece of pixel data obtained bycombining the eight pieces of adjacent pixel data).

In some embodiments of the present disclosure, a plurality of pieces ofpixel data may be combined into one piece of pixel data by using any ofthe following schemes.

In Scheme 1, an average of values of the plurality of pieces of pixeldata may be calculated to obtain a value of the resulting one piece ofpixel data. For example, each piece of pixel data may include fourvalues, R, G, G and B. In this case, an average of the R values of theplurality of pieces of pixel data may be calculated as an R value of theresulting one piece of pixel data; an average of the G values of theplurality of pieces of pixel data may be calculated as a G value of theresulting one piece of pixel data; and so on. FIG. 7G is a schematicdiagram showing four pieces of pixel data combined into one piece ofpixel data. Each bold black box represents one piece of pixel data. Eachpiece of pixel data includes four values, R, G, G and B. Four pieces ofadjacent pixel data are shown on the left and one piece of pixel dataobtained by combining the four pieces of adjacent pixel data is shown onthe right. The R value of the resulting one piece of data may be anaverage of the R values of the four pieces of adjacent pixel data, andthe same also applies to other values.

In Scheme 2, a sum of values of the plurality of pieces of pixel datamay be calculated to obtain a value of the resulting one piece of pixeldata. For example, each piece of pixel data may include four values, R,G, G and B. In this case, a sum of the R values of the plurality ofpieces of pixel data may be calculated as an R value of the resultingone piece of pixel data; a sum of the G values of the plurality ofpieces of pixel data may be calculated as a G value of the resulting onepiece of pixel data; and so on.

The above Scheme 1 is applicable in an environment having a goodlighting condition, while the above Scheme 2 is applicable in a darkenvironment. In an embodiment of the present disclosure, Scheme 1 orScheme 2 may be used for combining the plurality of pieces of pixel datadepending on the operating environment of the camera. For example,Scheme 1 may be used for combining the plurality of pieces of pixel datain an environment having a good lighting condition, or Scheme 2 may beused for combining the plurality of pieces of pixel data in a darkenvironment. It may be set as desired by those skilled in the art andthe present disclosure is not limited to this.

By combining the pixel data, the amount of data to be transmitted may bereduced and, additionally, in a low illumination situation, byaccumulating values of a plurality of pieces of pixel data using Scheme2, a quantum conversion rate of one single piece of pixel data may beincreased effectively, such that the sensitivity of the pixel data maybe improved. In a normal illumination situation, by summing andaveraging a plurality of pieces of adjacent pixel data using Scheme 1,the detail information of the image may be kept, without causingoverflow of pixel values.

Embodiment 2

According to Embodiment 2 of the present disclosure, a method for imagecapturing control is provided. The method is applicable to an imagecapturing controller (e.g., Field-Programmable Gate Array (FPGA),Central Processing Unit (CPU) or microprocessor). As shown in theflowchart of FIG. 8, the method includes the following steps.

At step 201, a position region in which a target object is to be locatedin a next frame of image is predicted based on an object tracking resultassociated with a previous frame of image, and the position region isdetermined as a target ROI corresponding to the next frame of image.

At step 202, ROI information of the target ROI corresponding to the nextframe of image is transmitted to a camera.

In some embodiments, the method process shown in FIG. 8 may furtherinclude steps 203-205, as shown in FIG. 9.

At step 203, pixel data of a target ROI corresponding to a current frameof image is received from the camera.

At step 204, an object detection is performed on the pixel data of thetarget ROI corresponding to the current frame of image, to obtain anobject detection result associated with the current frame of image.

At step 205, an object tracking result associated with the current frameof image is determined, based on the object detection result associatedwith the current frame of image and the object tracking resultassociated with the previous frame of image.

In some embodiments of the present disclosure, in the step 201, anexisting object tracking algorithm may be used to predict the positionregion in which each target object is to be located in the next frame ofimage, based on the object tracking result associated with the previousframe of image. The present disclosure is not limited to any specificimplementation of the step 201. FIG. 10A shows the to object trackingresult associated with the previous frame of image (assuming that ObjectA and Object B are included). FIG. 10B shows the position regions (shownas dashed boxes) of the respective target objects in the next frame ofimage as predicted based on the object tracking result associated withthe previous frame of image. The position regions are determined astarget ROIs corresponding to the next frame of image.

In some embodiments of the present disclosure, the ROI informationcontains information describing the position region in which the ROI islocated in the frame of image. For this, reference may be made to therelevant description in the above Embodiment 1 and details thereof willbe omitted here.

In some embodiments of the present disclosure, in the step 204, anexisting object detection algorithm may be used to obtain the objectdetection result associated with the current frame of image. The presentdisclosure is not limited to any specific implementation of the step204.

In particular, the step 205 may be implemented as follows. The objectdetection result associated with the current frame of image (the targetobject contained is referred to as detection target) is matched with theobject tracking result associated with the previous frame of image (thetarget object contained is referred to as tracking target), to obtainthe matched tracking target and detection target, or the unmatchedtracking target and detection target. The matched tracking target isprocessed with Kalman filter and feature fusion and then included in apreconfigured set, T_1. The unmatched tracking target is predicted withKalman filter or removed, to obtain a set, T_2. The unmatched detectiontarget is included in a preconfigured set, T_3. The union of the setsT_1, T_2 and T_3 is determined as the object tracking result associatedwith the current frame of image. FIG. 11A shows the object detectionresult associated with the current frame of image (including detectiontargets A, B and C). FIG. 11B shows the object tracking resultassociated with the previous frame of image (including tracking targetsA, B, D and E). FIG. 11C shows the object tracking result associatedwith the current frame of image (including tracking targets A, B, C andD).

Of course, in the above step 205, the object tracking result associatedwith the current frame of image may be obtained by those skilled in theart using any existing object tracking techniques. The presentdisclosure is not limited to any specific object tracking technique.

Embodiment 3

Based on the same concept as the method for image capturing according tothe above Embodiment 1, in an embodiment of the present disclosure, anapparatus for image capturing is also provided. The apparatus 1 forimage capturing is applicable to a camera. Referring to the structureshown in FIG. 12, the apparatus 1 for image capturing includes thefollowing units.

A parameter obtaining unit 11 is configured to obtain ROI information ofa target ROI corresponding to a current frame of image.

A pixel data receiving unit 12 is configured to buffer captured pixeldata of the current frame of image.

An obtaining unit 13 is configured to obtain, when determining that thepixel data of the current frame of image has been buffered completely,pixel data associated with the target ROI from the buffered pixel dataof the current frame of image based on the ROI information if the ROIinformation of the target ROI corresponding to the current frame ofimage may be obtained from the parameter obtaining unit 11, and transmitthe pixel data associated with the target ROI to a transmitting unit 14.

The transmitting unit 14 is configured to output the pixel dataassociated with the target ROI.

In some embodiments, the obtaining unit 13 may be configured to obtainthe pixel data associated with the target ROI from the buffered pixeldata of the current frame of image based on the ROI information by:determining the pixel data associated with the target ROI in the pixeldata of the current frame of image based on the ROI information;determining a storage address of the pixel data associated with thetarget ROI in a buffer; and reading the pixel data from the storageaddress to obtain the pixel data associated with the target ROI.

In some embodiments, the obtaining unit 13 may be further configured to,if the ROI information of the target ROI corresponding to the currentframe of image cannot be obtained from the parameter obtaining unit 11:combine n*m pieces of adjacent pixel data in the pixel data of thecurrent frame of image into one piece of pixel data sequentially, toobtain compressed pixel data associated with the current frame of image,and transmit the compressed pixel data to the transmitting unit 14,where n is a natural number divisible by a total number of rows in thecurrent frame of image, and m is a natural number divisible by a totalnumber of columns in the current frame of image. Accordingly, thetransmitting unit 14 may be further configured to output the compressedpixel data associated with the current frame of image.

In some embodiments of the present disclosure, for the specificimplementation of the obtaining unit 13, reference may be made to theabove description relevant to the steps 104 and 105 in Embodiment 1, anddetails thereof will be omitted here.

Embodiment 4

According to Embodiment 4 of the present disclosure, a camera isprovided. Referring to the structure shown in FIG. 13, the camera 2 mayinclude: an image sensor 21 configured to capture pixel data of acurrent frame of image in accordance with a predetermined cameraparameter and transmit the captured pixel data to an apparatus 1 forimage capturing; and the apparatus 1 for image capturing configured toobtain ROI information of a target ROI corresponding to the currentframe of image; receive the pixel data of the current frame of imagefrom the image sensor 21 and buffer the pixel data; obtain, whendetermining that the pixel data of the current frame of image has beenbuffered completely, pixel data associated with the target ROI from thebuffered pixel data of the current frame of image based on the ROIinformation if the ROI information of the target ROI corresponding tothe current frame of image may be obtained; and output the pixel dataassociated with the target ROI.

In some embodiments, the apparatus 1 for image capturing may be furtherconfigured to, if the ROI information of the target ROI corresponding tothe current frame of image cannot be obtained: combine n*m pieces ofadjacent pixel data in the pixel data of the current frame of image intoone piece of pixel data sequentially, to obtain compressed pixel dataassociated with the current frame of image, and output the compressedpixel data, where n is a natural number divisible by a total number ofrows in the current frame of image, and m is a natural number divisibleby a total number of columns in the current frame of image.

The apparatus 1 for image capturing may continuously receive the pixeldata transmitted from the image sensor 21 in accordance with aninterface timing sequence, and buffer the received pixel data in apreconfigured storage space with the resolution of the image sensor 21in accordance with row/column information in the timing sequence.

In Embodiment 4 of the present disclosure, for the apparatus 1 for imagecapturing, reference may be made to relevant description in Embodiment 3and details thereof will be omitted here.

Embodiment 5

Based on the same concept as the method for image capturing controlaccording to the above Embodiment 2, in an embodiment of the presentdisclosure, an apparatus for image capturing control is provided. Theapparatus 3 for image capturing control is applicable to an imagecapturing controller (e.g., FPGA, CPU or microprocessor). Referring tothe structure shown in FIG. 14, the apparatus 3 for image capturingcontrol includes the following units.

An ROI determining unit 31 is configured to predict a position region inwhich a target object is to be located in a next frame of image based onan object tracking result associated with a previous frame of image, anddetermine the position region as a target ROI corresponding to the nextframe of image.

A transmitting unit 32 is configured to transmit ROI information of thetarget ROI corresponding to the next frame of image to a camera.

In some embodiments, the apparatus 3 for image capturing control asshown in FIG. 14 may further include an image receiving unit 33, anobject detecting unit 34 and an object tracking unit 35, as shown FIG.15.

The image receiving unit 33 is configured to receive pixel data of atarget ROI corresponding to a current frame of image from the camera.

The object detecting unit 34 is configured to perform an objectdetection on the pixel data of the target ROI corresponding to thecurrent frame of image, to obtain an object detection result associatedwith the current frame of image.

The object tracking unit 35 is configured to determine an objecttracking result associated with the current frame of image based on theobject detection result associated with the current frame of image andthe object tracking result associated with the previous frame of image.

In Embodiment 5 of the present disclosure, the object detecting unit 34may use an existing object detection technique to perform the objectdetection on the pixel data of the target ROI, and details thereof willbe omitted here.

In Embodiment 5 of the present disclosure, the object tracking unit 35may use an existing object tracking technique to obtain the objecttracking result associated with the current frame of image, or mayperform the object tracking using the scheme described above inconnection with the step 205 in Embodiment 2, and details thereof willbe omitted here.

Embodiment 6

According to Embodiment 6 of the present disclosure, a system for imagecapturing is provided. The system for image capturing includes a camera2 and an apparatus 3 for image capturing control, as shown in FIG. 16.

The apparatus 3 for image capturing control is configured to predict aposition region in which a target object is to be located in a nextframe of image based on an object tracking result associated with aprevious frame of image, and determine the position region as a targetROI corresponding to the next frame of image; and transmit ROIinformation of the target ROI corresponding to the next frame of imageto the camera 2.

The camera 2 is configured to obtain ROI information of a target ROIcorresponding to a current frame of image from the apparatus 3 for imagecapturing control; capture pixel data of the current frame of image inaccordance with a predetermined camera parameter and buffer the pixeldata of the current frame of image; obtain, when determining that thepixel data of the current frame of image has been buffered completely,pixel data associated with the target ROI from the buffered pixel dataof the current frame of image based on the ROI information if the ROIinformation of the target ROI corresponding to the current frame ofimage may be obtained; and output the pixel data associated with thetarget ROI.

In some embodiments, the camera 2 may be further configured to, if theROI information of the target ROI corresponding to the current frame ofimage cannot be obtained: combine n*m pieces of adjacent pixel data inthe pixel data of the current frame of image into one piece of pixeldata sequentially, to obtain compressed pixel data associated with thecurrent frame of image, and output the compressed pixel data, where n isa natural number divisible by a total number of rows in the currentframe of image, and m is a natural number divisible by a total number ofcolumns in the current frame of image.

In Embodiment 6 of the present disclosure, for the specific structure ofthe apparatus 3 for image capturing control, reference may be made toEmbodiment 5 and details thereof will be omitted here. For the specificstructure of the camera 2, reference may be made to Embodiment 4 anddetails thereof will be omitted here.

In an embodiment of the present disclosure, an image data transmissionprotocol may be agreed in advance between the apparatus for imagecapturing in the camera 2 and the apparatus for image capturing control.Information on the format to be used by the apparatus for imagecapturing for transmitting pixel data of a target ROI corresponding to aframe of image to the apparatus for image capturing control may bespecified in the protocol. For example, when a frame of imagecorresponds to one piece of image data, the information on the formatmay specify which fields are included in the one piece of image data andthe order thereof. For example, one piece of image data may include thefollowing three fields sequentially: a field for indicating statementinformation of starting transmission of the current frame of image, afield for describing a number of target ROIs contained in the currentframe of image, and a field containing pixel data of all the target ROIscorresponding to the current frame of image.

The basic principles of the present disclosure have been described abovewith reference to the embodiments. However, it may be appreciated bythose skilled in the art that all or any of the steps or components ofthe method or apparatus according to the present disclosure may beimplemented in hardware, firmware, software or any combination thereofin any computing device (including a processor, a storage medium, etc.)or a network of computing devices. This may be achieved by those skilledin the art using their basic programing skills based on the descriptionof the present disclosure.

It may be appreciated by those skilled in the art that all or part ofthe steps in the method according to the above embodiment may beimplemented in hardware following instructions of a program. The programmay be stored in a computer readable storage medium. The program, whenexecuted, may include one or any combination of the steps in the methodaccording to the above embodiment.

Further, the functional units in the embodiments of the presentdisclosure may be integrated into one processing module or may bephysically separate, or two or more units may be integrated into onemodule. Such integrated module may be implemented in hardware orsoftware functional units. When implemented in software functional unitsand sold or used as a standalone product, the integrated module may bestored in a computer readable storage medium.

It can be appreciated by those skilled in the art that the embodimentsof the present disclosure can be implemented as a method, a system or acomputer program product. The present disclosure may include purehardware embodiments, pure software embodiments and any combinationthereof. Also, the present disclosure may include a computer programproduct implemented on one or more computer readable storage mediums(including, but not limited to, magnetic disk storage and opticalstorage) containing computer readable program codes.

The present disclosure has been described with reference to theflowcharts and/or block diagrams of the method, device (system) andcomputer program product according to the embodiments of the presentdisclosure. It can be appreciated that each process and/or block in theflowcharts and/or block diagrams, or any combination thereof, can beimplemented by computer program instructions. Such computer programinstructions can be provided to a general computer, a dedicatedcomputer, an embedded processor or a processor of any other programmabledata processing device to constitute a machine, such that theinstructions executed by a processor of a computer or any otherprogrammable data processing device can constitute means forimplementing the functions specified by one or more processes in theflowcharts and/or one or more blocks in the block diagrams.

These computer program instructions can also be stored in a computerreadable memory that can direct a computer or any other programmabledata processing device to operate in a particular way. Thus, theinstructions stored in the computer readable memory constitute amanufacture including instruction means for implementing the functionsspecified by one or more processes in the flowcharts and/or one or moreblocks in the block diagrams.

These computer program instructions can also be loaded onto a computeror any other programmable data processing device, such that the computeror the programmable data processing device can perform a series ofoperations/steps to achieve a computer-implemented process. In this way,the instructions executed on the computer or the programmable dataprocessing device can provide steps for implementing the functionsspecified by one or more processes in the flowcharts and/or one or moreblocks in the block diagrams.

While the embodiments of the present disclosure have described above,further alternatives and modifications can be made to these embodimentsby those skilled in the art in light of the basic inventive concept ofthe present disclosure. The claims as attached are intended to cover theabove embodiments and all these alternatives and modifications that fallwithin the scope of the present disclosure.

Obviously, various modifications and variants can be made to theembodiments of the present disclosure by those skilled in the artwithout departing from the spirit and scope of the present disclosure.Therefore, these modifications and variants are to be encompassed by thepresent disclosure if they fall within the scope of the presentdisclosure as defined by the claims and their equivalents.

What is claimed is:
 1. A method for image capturing, comprising:obtaining Region Of Interest (ROI) information of a target ROIcorresponding to a current frame of image; buffering captured pixel dataof the current frame of image; obtaining, when determining that thepixel data of the current frame of image has been buffered completely,pixel data associated with the target ROI from the buffered pixel dataof the current frame of image based on the ROI information if the ROIinformation of the target ROI corresponding to the current frame ofimage can be obtained; and outputting the pixel data associated with thetarget ROI.
 2. The method of claim 1, wherein said obtaining the pixeldata associated with the target ROI from the buffered pixel data of thecurrent frame of image based on the ROI information comprises:determining the pixel data associated with the target ROI in the pixeldata of the current frame of image based on the ROI information;determining a storage address of the pixel data associated with thetarget ROI in a buffer; and reading the pixel data from the storageaddress to obtain the pixel data associated with the target ROI.
 3. Themethod of claim 1, further comprising, subsequent to determining thatthe pixel data of the current frame of image has been bufferedcompletely, if the ROI information of the target ROI corresponding tothe current frame of image cannot be obtained: combining n*m pieces ofadjacent pixel data in the pixel data of the current frame of image intoone piece of pixel data sequentially, to obtain compressed pixel dataassociated with the current frame of image, and outputting thecompressed pixel data, where n is a natural number divisible by a totalnumber of rows in the current frame of image, and m is a natural numberdivisible by a total number of columns in the current frame of image. 4.The method of claim 2, further comprising, subsequent to determiningthat the pixel data of the current frame of image has been bufferedcompletely, if the ROI information of the target ROI corresponding tothe current frame of image cannot be obtained: combining n*m pieces ofadjacent pixel data in the pixel data of the current to frame of imageinto one piece of pixel data sequentially, to obtain compressed pixeldata associated with the current frame of image, and outputting thecompressed pixel data, where n is a natural number divisible by a totalnumber of rows in the current frame of image, and m is a natural numberdivisible by a total number of columns in the current frame of image. 5.A method for image capturing control, comprising: predicting a positionregion in which a target object is to be located in a next frame ofimage based on an object tracking result associated with a previousframe of image, and determining the position region as a target Regionof Interest (ROI) corresponding to the next frame of image; andtransmitting ROI information of the target ROI corresponding to the nextframe of image to a camera.
 6. The method of claim 5, furthercomprising: receiving pixel data of a target ROI corresponding to acurrent frame of image from the camera; performing an object detectionon the pixel data of the target ROI corresponding to the current frameof image, to obtain an object detection result associated with thecurrent frame of image; and determining an object tracking resultassociated with the current frame of image based on the object detectionresult associated with the current frame of image and the objecttracking result associated with the previous frame of image.
 7. Anapparatus for image capturing, comprising: a parameter obtaining unitconfigured to obtain Region Of Interest (ROI) information of a targetROI corresponding to a current frame of image; a pixel data receivingunit configured to buffer captured pixel data of the current frame ofimage; an obtaining unit configured to obtain, when determining that thepixel data of the current frame of image has been buffered completely,pixel data associated with the target ROI from the buffered pixel dataof the current frame of image based on the ROI information if the ROIinformation of the target ROI corresponding to the current frame ofimage can be obtained from the parameter obtaining unit, and transmitthe pixel data associated with the target ROI to a transmitting unit;and the transmitting unit configured to output the pixel data associatedwith the target ROI.
 8. The apparatus of claim 7, wherein the obtainingunit is configured to obtain the pixel data associated with the targetROI from the buffered pixel data of the current frame of image based onthe ROI information by: determining the pixel data associated with thetarget ROI in the pixel data of the current frame of image based on theROI information; determining a storage address of the pixel dataassociated with the target ROI in a buffer; and reading the pixel datafrom the storage address to obtain the pixel data associated with thetarget ROI.
 9. The apparatus of claim 7, wherein the obtaining unit isfurther configured to, if the ROI information of the target ROIcorresponding to the current frame of image cannot be obtained from theparameter obtaining unit: combine n*m pieces of adjacent pixel data inthe pixel data of the current frame of image into one piece of pixeldata sequentially, to obtain compressed pixel data associated with thecurrent frame of image, and transmit the compressed pixel data to thetransmitting unit, where n is a natural number divisible by a totalnumber of rows in the current frame of image, and m is a natural numberdivisible by a total number of columns in the current frame of image,and the transmitting unit is further configured to output the compressedpixel data associated with the current frame of image.
 10. The apparatusof claim 8, wherein the obtaining unit is further configured to, if theROI information of the target ROI corresponding to the current frame ofimage cannot be obtained from the parameter obtaining unit: combine n*mpieces of adjacent pixel data in the pixel data of the current frame ofimage into one piece of pixel data sequentially, to obtain compressedpixel data associated with the current frame of image, and transmit thecompressed pixel data to the transmitting unit, where n is a naturalnumber divisible by a total number of rows in the current frame ofimage, and m is a natural number divisible by a total number of columnsin the current frame of image, and the transmitting unit is furtherconfigured to output the compressed pixel data associated with thecurrent frame of image.
 11. An apparatus for image capturing control,comprising: a Region of Interest (ROI) determining unit configured topredict a position region in which a target object is to be located in anext frame of image based on an object tracking result associated with aprevious frame of image, and determine the position region as a targetROI corresponding to the next frame of image; and a transmitting unitconfigured to transmit ROI information of the target ROI correspondingto the next frame of image to a camera.
 12. The apparatus of claim 11,further comprising: an image receiving unit configured to receive pixeldata of a target ROI corresponding to a current frame of image from thecamera; an object detecting unit configured to perform an objectdetection on the pixel data of the target ROI corresponding to thecurrent frame of image, to obtain an object detection result associatedwith the current frame of image; and an object tracking unit configuredto determine an object tracking result associated with the current frameof image based on the object detection result associated with thecurrent frame of image and the object tracking result associated withthe previous frame of image.
 13. A camera, comprising: an image sensorconfigured to capture pixel data of a current frame of image inaccordance with a predetermined camera parameter and transmit thecaptured pixel data to an apparatus for image capturing; and theapparatus for image capturing configured to obtain Region Of Interest(ROI) information of a target ROI corresponding to the current frame ofimage; receive the pixel data of the current frame of image from theimage sensor and buffer the pixel data; obtain, when determining thatthe pixel data of the current frame of image has been bufferedcompletely, pixel data associated with the target ROI from the bufferedpixel data of the current frame of image based on the ROI information ifthe ROI information of the target ROI corresponding to the current frameof image can be obtained; and output the pixel data associated with thetarget ROI.
 14. The camera of claim 13, wherein the apparatus for imagecapturing is further configured to, if the ROI information of the targetROI corresponding to the current frame of image cannot be obtained:combine n*m pieces of adjacent pixel data in the pixel data of thecurrent frame of image into one piece of pixel data sequentially, toobtain compressed pixel data associated with the current frame of image,and output the compressed pixel data, where n is a natural numberdivisible by a total number of rows in the current frame of image, and mis a natural number divisible by a total number of columns in thecurrent frame of image.
 15. A system for image capturing, comprising: anapparatus for image capturing control configured to predict a positionregion in which a target object is to be located in a next frame ofimage based on an object tracking result associated with a previousframe of image, and determine the position region as a target Region ofInterest (ROI) corresponding to the next frame of image; and transmitROI information of the target ROI corresponding to the next frame ofimage to a camera; and the camera configured to obtain ROI informationof a target ROI corresponding to a current frame of image from theapparatus for image capturing control; capture pixel data of the currentframe of image in accordance with a predetermined camera parameter andbuffer the pixel data of the current frame of image; obtain, whendetermining that the pixel data of the current frame of image has beenbuffered completely, pixel data associated with the target ROI from thebuffered pixel data of the current frame of image based on the ROIinformation if the ROI information of the target ROI corresponding tothe current frame of image can be obtained; and output the pixel dataassociated with the target ROI.
 16. The system of claim 15, wherein thecamera is further configured to, if the ROI information of the targetROI corresponding to the current frame of image cannot be obtained:combine n*m pieces of adjacent pixel data in the pixel data of thecurrent frame of image into one piece of pixel data sequentially, toobtain compressed pixel data associated with the current frame of image,and output the compressed pixel data, where n is a natural numberdivisible by a total number of rows in the current frame of image, and mis a natural number divisible by a total number of columns in thecurrent frame of image.